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ABSTRACT 

This paper provides a model of course-embedded assessment for use in an undergraduate 
Accounting Information Systems course, and reports the results obtained from implementation. 
The profession’s educational objectives are mapped to specific computer skills and assignments, 
to provide direct evidence of learning outcomes. Indirect evidence of learning is provided by data 
collected with computer self-efficacy instrumentation. The paper addresses an important issue for 
persons entering the profession, in that they may have learned to use specific computer 
applications, but may lack the confidence in their own ability to use these skills. This may impact 
their ability to gain employment, or to maintain employment in accounting. The data collected 
may be employed to assess how well an AIS course supports the mission of its school, by 
demonstrating the acquisition of IT skills and the necessary self-efficacy to use those skills. 


I. INTRODUCTION 

s proficient computer use has come to be expected in accounting jobs, students’ beliefs in their 
ability to use computer technologies effectively, and their attitudes toward computers, have become 
critical factors in their professional success. Social cognitive theory (Bandura, 1977, 1997) posits 
that an individual’s “perceived self-efficacy refers to beliefs in one’s power to produce given levels of attainment. A 
self-efficacy assessment (by an individual) therefore includes both the affirmation of capability and the strength of 
that belief’ (Bandura, 1997:382). An individual’s self-efficacy can affect their interest in learning about computer 
systems (Stone, Arunachalam, and Chandler, 1996); their proficiency in using computers (Davis, Bagozzi, and 
Warshaw, 1989; Mills, 1997; Walters and Necessary, 1996) and their ability to obtain or maintain employment 
(Walters and Necessary, 1996). Stoner notes the effect this can have on students: “All too often students fail to 
thrive in computer-based work, not because of a lack of skills but because of a lack of confidence in the skills they 
have. ” (Italics added. 1999:234) 

In spite of this attention, there is a small body of work examining the role of self-efficacy in computing 
behavior (Burkhardt and Brass, 1990; Compeau, Higgins, and Huff, 1999; Gist, Schwoerer, and Rosen, 1989; Henry 
and Stone, 2003; Johnson and Marakas, 2000; Kinsie, Delcourt, and Powers, 1994; and Martoccio and Judge, 1997). 
To our knowledge, there is no work concerning the use of self-efficacy as an assessment tool. 

This paper addresses one of the main issues in why some students are not successful in computer-based 
work, by incorporating computer self-efficacy measurement along with instruction and assignments in four computer 
applications in an undergraduate Accounting Information Systems (AIS) course. It employs an instrument for 
application-specific changes in students’ willingness to use what they learned in class (computer self-efficacy). 
Together the steps outlined in the paper provide a course-embedded assessment plan, with both direct and indirect 
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evidence of learning outcomes, to offer a method of assessing the class’s contribution to departmental and school 
objectives. A number of papers on computer self-efficacy use short time periods over which to measure self- 
efficacy; this paper tested changes in computer self-efficacy over a three-month period. 

The remainder of the paper is organized as follows; Section II discusses assessment plans and self-efficacy. 
Section III describes the design of the study. Section IV presents the results obtained and Section V contains the 
conclusions. 

II. ASSESSMENT PLANS AND SELF-EFFICACY 

Developing assessment plans to incorporate behavioral objectives, computer skills, and computer self-efficacy 

In this section we describe how our plan satisfies two complementary goals of accounting assessment. It 
provides two different course-embedded assessments, so that the assessment measures current computer skills, and it 
maps the profession’s behavioral objectives of accounting education into discrete, measurable computer assignments. 
Secondly, it incorporates the important construct of self-efficacy into a separate pretest-posttest assessment of 
students’ beliefs in their abilities to use the computer skills they have just been taught. Self-efficacy “...reflects an 
individual’s confidence in his/her ability to perform the behavior required to produce specific outcomes and is 
thought to directly impact the choice to engage in a task, as well as the effort that will be expended and the 
persistence that will be exhibited” (Kinzie, Delcourt, and Powers, 1994:747). 

First we discuss the use of behavioral objectives as the foundation for the plan, followed by the role of self- 
efficacy in assessment. 

Among the objectives of the AIS class are four which have been adopted from the Accounting Education 
Change Commission. The AECC promulgated a series of objectives for the education of accountants which were 
classified as cognitive, affective, or behavioral. The AECC’s definition of behavioral objectives suggests that there 
are numerous skills capable of being taught with or augmented with the use of computers and related technology: 

Behavioral objectives generally include some of the so-called "softer" skills, and encompass the following: critical- 
thinking skills, problem-solving ability, effective writing, effective oral communication, including both listening and 
speaking skills, quantitative analysis, leadership skills, team-building skills, computer literacy, library and 
information-technology competence, and values awareness (Baker et al., 1994:107). 

The assessment plan related the profession’s relevant educational objectives to the types of tasks that 
accountants perform with computers, and to using typical software to accomplish these tasks. Table 1 begins (from 
the left) with the behavioral objectives from the AECC’s Position Statement No. 1 we believed relevant to the AIS 
course. For each unobservable objective there follows a skill to represent it. Students completed one or two 
assignments for a grade in each of these to demonstrate what had been learned. The example software column lists 
the programs and reference databases students received training on and used to complete assignments. Finally the 
skills given in the righthand column agree with the computer skill grouping from the American Accounting 
Association’s Comprehensive List of Soft Skills (Calderon, Green, and Harkness, 2003:7). 

Students graduating from our program go to work as entry-level accountants. The types of assignments 
noted in Table 1 were intended to simulate professional work assignments. In a survey of over 4,000 CPAs, Prawitt, 
Romney, and Zarowin (1997) reported that 98% of CPAs used spreadsheets, 52% used presentation graphics 
programs, and about 50% used a database program in their practice work. The products they used were 
overwhelmingly from the Microsoft Office Professional suite (Excel, Access, and PowerPoint). We used the same 
products these practitioners did, and we added the use of research or reference databases such as are found in 
university libraries or a larger CPA/consulting firm. These included Lexis/Nexis, Thomson Research, ABI/Inform, 
and Mergent Online. We did not teach nor test the use of tax research databases nor accounting general ledger 
programs, as these are taught in other accounting courses. 
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I Table 1. Objectives of accounting education, computer skills, and assignments. 1 

Behavioral Representative Skills 

Objectives 

Assignments/ 
Assessment Tasks 

Example Software 

Computer Skill 

Communication Ability to present, discuss, 

skills & defend views 

effectively 

Flowcharting 
internal controls 

Case presentations 

M/S PowerPoint 

M/S Word 

Using Presentation 
Graphics 

Communication Ability to locate, obtain, 

skills organize, report, & use 

information from human, 
print, & electronic sources 
(Library & IT competence) 

Merger & 
acquisition case 

Article reviews of 
current topics 

Library databases (e.g., 
Lexis/Nexis, Mergent 
Online, Proquest, S&P 
Net Ad vantage) 

Internet 

Metacrawler 

Online Research 
Queries 

Accounting Identify, summarize, & 

knowledge analyze financial data in 

business organizations 

Merger & 
acquisition case 
PivotTables 

M/S Excel 

Using Spreadsheet 
Tools 

Accounting Concepts & principles of 

knowledge information system design 

& use 

Construct a simple 
accounting system 
using a relational 
database 

M/S Access 

Using Database 

Tools 


Students received training in the four computer skills given in Table 1, combining enactive mastery (hands- 
on) training and behavior modeling precepts (discussed under self-efficacy). The assignments in Table 1 and 
discussed in the results section were predominantly completed in class. The instructor critiqued each, offering 
comment and suggestions if needed. Students then made improvements or corrections and submitted their work for a 
grade. This sequence provided direct evidence that one of the course’s objectives had been satisfied. Figure 1 
below depicts on a timeline the six components of the assessment plan. The components numbered two through five 
represent this sequence beginning with hands-on instruction and ending with the assignment of grades. 

Figure 1 shows as the first and last components the collection of data (pretest and posttest) regarding 
students’ assessment of their self-efficacy with respect to each of the four computer skills they learned in the class. 
The pretest and posttest computer self-efficacy questionnaires represent indirect evidence of a learning outcome. For 
purposes of accreditation, direct evidence is preferred, but it may be necessary to collect indirect evidence about the 
course or program being assessed (Bayes, 2004). 


Figure 1. Model of Course-Embedded Assessment 

Sequence of assessment plan components 

1 2 3 4 5 6 


1 . 

2 . 

3 . 

4 . 

5 . 

6 . 


Pretest _ 

Software training 

Assignments - 

Critique, return for changes 
Assignments graded 
Posttest _ 


Direct Evidence 


*■ 


r -\ 

Indirect 

Evidence 

\ _ / 


Self-Efficacy 

As Figure 1 indicates, we used a computer self-efficacy assessment instrument at the beginning of the class, 
prior to any software training or computer work, and again at the end of the class to collect evidence as to whether or 
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not students’ assessment of their computer self-efficacy had changed. Using self-efficacy changes to assess the 
effectiveness of the AIS class provides indirect evidence of learning outcomes, as students judge that they can 
accomplish more with these computer skills than they believed possible at the start of the class. 

The impetus for this approach came from recent research into the effects that the increased emphasis on 
computers and information technology has on accounting education (David, McCracken and Reckers, 2003). These 
changes will likely affect students’ perceptions of their accounting education. Stone, Arunachalam and Chandler 
observed that “...of potentially equal importance, are the effects of changes in accounting education on students’ 
beliefs in their ability to accomplish accounting related tasks (i.e., their “accounting-related self-efficacy”) and 
students’ feelings about computers and computer-based technology” (1996:346). Walters and Necessary (1996) 
have shown a relationship between individuals’ perceptions of the importance of technology and proficiency in using 
technology. Negative attitudes towards computers may stop individuals from gaining access to or effectively using 
computers in the workplace and may even limit their opportunities for getting and holding employment (Walters et 
ah, 1996). 

Accounting educators expect routine computer use by students. However, the instructors are also shaping 
attitudes that will affect their students’ control over technology after graduation. Thus, it is necessary to both 
enhance and measure students’ self-efficacy with respect to computer technologies (Ertmer, Evenbeck, Cennamo, 
and Lehman, 1994; Kinzie et ah, 1994). Because of the importance of information technology proficiency in the 
accounting profession, students’ perceptions of confidence towards technology are critical to ensure their successful 
utilization of technology and related software (Mills, 1997). 

Albert Bandura, founder of the self-efficacy concept, explained the idea thusly: “Perceived self-efficacy 
refers to beliefs in one’s power to produce given levels of attainment. A self-efficacy assessment, therefore, includes 
both the affirmation of capability and the strength of that belief’ (1997:382). Individuals with high self-efficacy will 
put more effort into a task; they will persevere longer; and they will realize higher levels of accomplishment than 
individuals with lower self-efficacy. 

Information about a person’s self-efficacy has four sources (Bandura, 1997), the first two of which are 
relevant to this discussion. These are enactive mastery experiences, vicarious experience, verbal persuasion, and 
physiological and affective states. Enactive mastery (hands-on) experiences are actual proof that an individual can 
succeed at a task, and so are the strongest source. As students experience more successes with a computer, their 
belief in self-efficacy will generally increase. Consequently, actual hands-on training serves two purposes - students 
can build up confidence in their ability as well as learn the skill itself. If they are taught the computer skills in the 
AIS class, where they may ask questions and be coached, they will likely have a successful experience, and this will 
strengthen their self-efficacy. It may also reduce the amount of unsuccessful efforts, which would reduce self- 
efficacy, discouraging them from using the technology (Compeau and Higgins, 1995). 

The second source of self-efficacy information is vicarious experience. In a classroom, a student seeing 
someone else modeling the desired behavior will believe that he or she can also perform successfully (Bandura, 
1997:87). To be effective as a learning tool, the greater the similarity between the model and the student, the more 
effective will be the experience. This suggests that having another student model successful computer work is more 
useful than having an instructor alone model the behavior. 

III. DESIGN AND INSTRUMENTATION 

The research design is a one group, two measurement protocol in which all students receive the treatment. 
Pretest and posttest questionnaires were based on the Self-Efficacy for Computer Technologies instrument (SCT) 
(Kinzie et ah, 1994). The SCT measures confidence in using a number of applications, including spreadsheets, 
databases, and library reference databases (Which may be either online or on CD-ROM). An additional scale was 
developed by the authors to include presentation graphics programs. One objective of the class is to teach and 
improve students’ abilities in using applications that are related to accounting work, so an instrument that focused on 
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application-specific computer self-efficacy (CSE) was appropriate, as compared to instruments which measure 
overall computer self-efficacy (Johnson and Marakas, 2000). For each computer skill there is a scale with seven to 
nine items, all of the form “1 feel confident...’’and then requesting a response to the specific item, such as 
“...selecting search terms for a database literature search.” Possible responses are on a four-point Likert and include 
these choices (point values in parentheses): “strongly disagree” (value = 1); “somewhat disagree” (value = 2); 
“somewhat agree” (value = 3); and “strongly agree” (value = 4). 

The internal consistency reliability estimates (Cronbach’s alpha) for each of the scales are: Electronic 
Spreadsheets, 0.9015; Database, 0.9763; Library Reference Databases, 0.9767; and Presentation Graphics, 0.9721. 
These first three scores agree with those reported by Kinsie et al. (1994). The new scale for presentation graphics 
has a similar high alpha score. Hair, Anderson, Tatham, and Black (1998:118) suggest the lower acceptable limit for 
this test is .70. A Bonferroni adjustment of the confidence level was used in the paired variable means tests, to 
protect against multiple testing. 

IV. RESULTS 

Several factors must be mentioned as potential limitations on this research. Since we cannot control outside 
influences affecting students’ learning of computer technologies (e.g., in other classes, on the job, or at home) we 
cannot prove that the assignments were solely responsible for the improvements in self-efficacy. However, there are 
few reasons for the students to have or use many of the applications. Examples of these are database and 
spreadsheets, because these are more professional tools rather than recreational tools or communications devices. 
The improvements on the library reference databases scale likewise cannot be proven to be only a result of the 
project, because this reference tool is widely available in local county libraries and the university library, and is 
bundled as a reference with many new computer sales. However, the question was worded to elicit responses to 
business-oriented databases. 

A limitation of the design is the lack of a control group. Due to the small number of students participating, 
there were inadequate numbers enrolled to allow for this feature. 

General ledger and tax programs are used in other classes and were not included in this research. 

Demographic Data 

The students who participated in this survey were attending a small public university, and were all business 
school students. Most entered university directly from high school. Ages ranged from 19 to 47 (mean = 23 years). 
The gender split was 32.5 percent males (n=37); 67.5 percent females (n=77). These 114 students were enrolled in a 
junior-level accounting information systems course, and completed both a pretest and posttest survey, of a total of 
176 students who completed the class. This yielded a 65% participation rate. Sixty-six percent were juniors, 33 
percent were seniors, and the balance was post-degree students. Virtually all were accounting majors. The survey 
covers eight semesters’ administration of this instrument. 

Students reported computer usage ranging from one to twenty years. The mean usage of computers was 8.7 
years for men, and 9.7 years for women with an overall mean of 9.4 years. Table 2 provides data on students’ 
computer and parental background. A majority of the students reported they had used a computer both at home and 
at work. They also reported they had pre-high school experience using computers, and that their parents used a 
computer on the job. 
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| Table 2. Computer Use, Experience, and Education. 1 

Variable 

Yes 

No 

N 

Do you use a computer at home? 

79.8% 

20.2% 

114 i 

Do you use a computer on your job? 

71.7% 

28.3% 

113 

Did you have any classroom computer experience prior to high school? 

69.3% 

30.7% 

114 

Does either of your parents use a computer on his or her job? 

64.9% 

35.1% 

114 

Do either of your parents have a college degree? 

43.0% 

57.0% 

114 

Do you already have a college degree? 

8.8% 

91.2% 

114 


Experience with Computer Technologies 

As a preliminary measure, students were asked on the pretest to describe their experience in using the four 
computer technologies that were to be taught. Table 3 below shows that many students used the technologies in high 
school. Spreadsheets were encountered in high school more often than the other products. A majority of the students 
first used databases, graphics, and library reference databases in college. The number of responses varies in the 
following tables due to non-responses. 


Table 3, First experience with computer technologies. 


Variable 

First used before 
high school 

First used in 
high school 

First used in 
college 

First used 
at home 

First used 
on a job 

N 

Spreadsheet 

4.5% 

48.6% 

39.6% 

0.9% 

6.3% 

111 

Database 

6.0% 

33.3% 

46.0% 

0% 

12.0% 

100 

Library databases 

6.5% 

30.4% 

52.2% 

7.6% 

3.3% 

92 

Graphics 

3.0% 

32.7% 

56.4% 

3.0% 

5.0% 

101 


Frequency of use 

A paired variable t-test of mean differences was performed on the pretest and posttest frequency of use data 
to determine if there was a difference in students’ frequency of use over the semester. The result was that for three of 
the four technologies surveyed, students’ frequency of usage increased significantly. (The values assigned for 
scoring were: several times a week = 1; once a week =2, 1-3 times a month = 3, and do not use = 4.) As shown in 
Table 4, students believed that their usage of all four technologies increased, and these were significant changes for 
all but library reference databases. 


! Table 4. Paired variables t-test for changes in frequency of use. ! 

Variable 

Pretest Mean 

Posttest Mean 

N 

2-tail Significance 

Use Spreadsheets 

1.98 

1.56 

105 

.000*** 

Use Relational Database Management Systems 

2.73 

2.02 

103 

.000*** 

Use Library Reference Databases 

2.71 

2.50 

101 

.088 

Use Graphics Programs 

2.82 

2.24 

103 

.000*** 


*Significant at .05; **significant at .01; ***significant at .001 


We next report on the results of the Self-Efficacy for Computer Technologies survey. This contains four 
scales measuring students’ comfort in using common business applications. The items responded to are listed at the 
left of each scale, and the 2-tailed significance of the t-statistic is in the box opposite the item. Using a two-tailed 
paired variable test of mean differences, with significance at the .05 level or less, the results obtained were as 
follows: The scale is 1 = strongly disagree, 2 = slightly disagree, 3 = slightly agree, and 4 = strongly agree. 
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Spreadsheets are used for many purposes in the profession. Increasing students’ ability to use this versatile 
and powerful program requires considerable practice. Students completed a web-based tutorial, participated in in- 
class exercises, produced financial statements in a spreadsheet, and used the PivotTable function to analyze financial 
data. Stronger students were engaged by the PivotTable and its analytic capability, and produced well-supported 
analyses, while the weaker ones had difficulty with the assignment. 

Significant results were obtained on the more involved spreadsheet functions (shown below). The two 
functions not showing a significant improvement (printing and saving a spreadsheet file) are possibly due to the 
Windows file menu used on the Office suite of programs: students have had considerable prior experience with these 
functions. 


i Table 5. Electronic Spreadsheets Scale: “I feel confident... j 

Variable 

Pretest Mean 

Posttest Mean 

N 

H 

...formatting the rows and columns of a spreadsheet” 

3.68 

3.84 

114 

.002** 

...naming the columns and rows in a spreadsheet” 

3.68 

3.84 

114 

.010** 

...entering appropriate formulas for calculation in a 
spreadsheet” 

3.40 

3.66 

114 

.000*** 

.. .entering data in a spreadsheet” 

3.74 

3.83 

114 

.047* 

...editing previous spreadsheet files” 

3.54 

3.80 

114 

.000*** 

.. .printing out the spreadsheet” 

3.82 

3.87 

114 

.241 

...saving a spreadsheet file” 

3.87 

3.89 

114 

.657 


*significant at .05; **significant at .01; ***significant at .001 


Database Management System Scale 

The database scale examines students’ attitudes towards using a database program. The assignments 
include readings, exercises, and the construction of a simple accounting system. In doing this they learn 
documentation skills, writing queries, and the implementation of internal controls. 

The database scale contains seven items. Results obtained show significant improvements in self-efficacy in 
all seven areas: 


Table 6. Database Management System Scale: “I feel confident... 

Variable 

Pretest 

Mean 

Posttest 

Mean 

N 

2-tail Sig. 

...formulating the data fields in a database” 

2.65 

3.34 

114 

.000*** 

...naming data fields in a database” 

2.74 

3.46 

114 

.000*** 

.. .entering records in a database” 

2.78 

3.45 

113 

.000*** 

...searching records in a database with specific terms” 

2.71 

3.30 

112 

.000*** 

...sorting records in a database” 

2.69 

3.30 

114 

.000*** 

.. .printing out records in a database” 

2.94 

3.56 

114 

.000*** 

...saving database files” 

3.01 

3.57 

113 

.000*** 


♦significant at .05; **significant at .01; ***significant at .001 
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The ability to identify and use information resources is paramount in the accounting profession. Training in 
this area was conducted by a reference librarian, who was also available to help students in the library. Students 
were assigned to research a proposed merger and acquisition case in the transportation industry. They used various 
features of the Thomson Research financial statements database (for individual companies’ text-searchable 
statements), the Standard & Poor’s Net Advantage database (for industry-specific commentary and analysis), and 
Lexis/Nexis (for regulatory data). They also were assigned article reviews to use in learning the use of periodicals 
databases, including ABI/Inform and Lexis-Nexis. Students ’ self-efficacy with respect to using library databases as 
research and reference tools showed significant improvements in all areas. 


| Table 7. Library Reference Databases Scale: “I feel confident... j 

Variable 

Pretest 

Mean 

Posttest 

Mean 

N 

2-tail Sig. 

.. .using a database on compact disc, such as Compact Disclosure, ABI 
Inform/Proquest, EconLit, Eric, etc.” 

2.42 

2.78 

112 

.001*** 

...selecting the right database on compact disc and starting a literature search” 

2.46 

2.87 

112 

.000*** 

.. .selecting search terms for a database literature search” 

2.59 

2.93 

112 

.001*** 

...getting into a database on compact disk and starting a literature search” 

2.52 

2.91 

112 

.000*** 

...using descriptors from a database literature search to obtain new search 
items” 

2.35 

2.85 

110 

.000*** 

.. .using the print function in a database search on compact disc” 

2.67 

3.10 

111 

.000*** 

...selecting the right terms for a database financial reports search” 

2.36 

2.88 

112 

.000*** 

...getting into a database on compact disk and starting a financial reports 
search” 

2.32 

2.86 

111 

.000*** 

...using descriptors from a database financial reports search to obtain new 
search items” 

2.27 

2.91 

107 

.000*** 


*significant at .05; **significant at .01; ***significant at .001 


Graphics Program Scale 

A graphics scale was constructed to test self-efficacy with respect to presentation graphics software. On 
entering the accounting workforce, they will need to be able to present ideas to their colleagues and managers, and 
this extends their communication skills. Assignments completed included flowcharting internal controls and case 
presentations. Students’ self-efficacy with respect to graphics packages showed significant improvement across all 
variables (below). 


Table 8. Graphics Scale: “I feel confident... 

Variable 

Pretest Mean 

Posttest Mean 

N 

2-tail Sig. 

...formatting the workspace in a graphics program” 

2.58 

3.28 

113 

.000*** 

.. .choosing a symbol to represent an action or object” 

2.72 

3.37 

113 

.000*** 

.. .placing the symbol on the workspace” 

2.77 

3.42 

112 

.000*** 

...linking several symbols together” 

2.60 

3.38 

112 

.000*** 

.. .adding text to a drawing” 

2.83 

3.45 

113 

.000*** 

.. .printing out the graphic image” 

2.96 

3.55 

113 

.000*** 

...saving the graphic file” 

2.96 

3.58 

113 

.000*** 


significant at .05; **significant at .01; ***significant at .001 
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This study demonstrated that course-embedded assessment in the AIS course can provide both direct and 
indirect evidence of learning. Focusing on improving students’ information technology self-efficacy may help to 
solve the issue of students who have learned the computer skills, but lack the confidence to use them. Removing this 
obstacle may be as important to their future success as teaching them specific computer skills. 
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